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ABSTRACT 

Aerospace  applications  require  precise  control  despite  uncertain  operating  conditions  and  unantici¬ 
pated  circumstances  such  as  battle  damage.  These  systems  must  be  designed  to  perform  robustly, 
despite  uncertain  design  models  and  difficult  to  analyze  nonlinear  effects.  They  must  be  capable  of 
learning  and  adapting  when  accumulating  data  indicates  that  previous  models  must  be  abandoned 
and  that  existing  control  strategies  must  be  changed.  Data-driven  design  methods,  collectively  known 
as  un-falsified  control  theory,  facilitate  the  creation  of  robust  control  systems  that  learn,  discover  and 
evolve  in  real  time  in  order  to  rapidly  switch  controller  gains  to  compensate  for  the  effects  of  battle, 
equipment  failures,  and  other  changing  circumstances.  Applications  studies  will  be  presented  that 
include  adaptive  robot  arm  control  and  missile  control. 

“I  have  devised  seven  separate  explanations,  each  of  which  would  cover  the  facts  as  far  as 
we  know  them.  But  which  of  these  is  correct  can  only  be  determined  by  fresh  information 
which  we  shall  no  doubt  find  waiting  for  us.  ” 

Sherlock  Holmes 
Arthur  Conan  Doyle 


1.0  INTRODUCTION 

Though  the  robust  multivariable  control  theory  that  has  evolved  over  the  past  quarter  century  offers 
a  major  improvement  over  earlier  algebraic  and  optimal  control  methods,  it  cannot  produce  reliable 
control  designs  unless  reliable  prior  bounds  on  plant  uncertainty  are  available.  This  applies  to  methods 
based  on  the  H ^  fi/Km- synthesis,  and  BMI/LMI/IQC  theories  [1]— [6] .  These  robust  control  design 
methods  all  have  an  Achilles  heel:  They  are  dependent  of  the  premise  that  uncertainty  models  are 
reliable,  and  they  offer  little  guidance  in  the  event  that  experimental  data  either  invalidates  prior 
knowledge  of  uncertainty  bounds  or,  perhaps,  provides  evidence  of  previously  unsuspected  patterns  in 
the  data.  That  is,  the  standard  H ^  fjc/Km- synthesis,  and  BMI/LMI/IQC  robust  control  techniques 
fail  in  the  all  too  common  situation  in  which  prior  knowledge  is  poor  or  unreliable. 

To  correct  this,  reliable  data-driven  adaptive  design  techniques  are  needed.  Ideally,  these  techniques 
should  incorporate  mechanisms  for  evaluating  the  design  implications  of  each  new  experimental  data 
point,  and  for  directly  integrating  that  information  into  the  mathematics  of  the  robust  control  design 
process  to  allow  methodical  update  and  re-design  of  control  strategies  so  as  to  accurately  reflect  the 
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implications  of  new  or  evolving  experimental  data.  Examples  of  recent  thrusts  in  this  direction  are  indi¬ 
rect  controller  tuning/adaptation  methods  based  on  control-oriented  identification  theory  and  [7]— [26] 
and,  more  recently,  related  direct  methods  that  bypass  plant  identification  based  on  controller  unfalsi¬ 
fication  [27] — [55] .  While  both  control-oriented  identification  theory  and  unfalsified  control  theory  are 
concerned  with  the  difficult  problem  of  assimilating  real-time  measurement  data  into  the  otherwise 
introspective  process  of  robust  control  design,  the  unfalsified  control  approach  is  a  particular  interest 
because  it  directly  and  precisely  characterizes  the  control  design  implications  of  experimental  data. 


2.0  DATA-DRIVEN  ROBUST  CONTROL 

Validation  —  or  more  precisely  unfalsification  —  of  hypotheses  against  physical  data  is  the  central 
aspect  of  the  process  of  scientific  discovery.  This  validation  process  allows  scientists  to  sift  the  elegant 
tautologies  of  pure  mathematics  in  order  to  discover  mathematical  descriptions  of  nature  that  are  not 
only  for  logically  self-consistent,  but  also  consistent  with  physically  observed  data.  This  data-driven 
process  of  validation  is  also  a  key  part  engineering  design.  Successful  engineering  design  techniques 
inevitably  arrive  at  a  point  where  pure  introspective  theory  and  model-based  analyses  must  be  tested 
against  physical  data.  But,  in  control  engineering  in  particular,  the  validation  process  is  one  that  has 
been  much  neglected  by  theoreticians.  Here,  the  theory  tying  control  designs  to  physical  data  has 
for  the  most  part  focused  on  pre-control-design  ‘system  identification’.  Otherwise,  the  mathematiza- 
tion  of  the  processes  of  post-design  validation  and  re-design  has  remained  relatively  unexplored  virgin 
territory.  In  particular,  a  satisfactory  quantitative  mathematical  theory  for  direct  feedback  of  experi¬ 
mental  design-validation  data  into  the  control  design  process  has  been  lacking,  though  this  seems  to 
be  changing  with  the  recent  introduction  of  a  theory  of  unfalsified  control  [31]. 

2.1  Theory:  Validation  and  Unfalsification 

Unfalsified  control  is  essentially  a  data-driven  adaptive  control  theory  that  permits  learning  based  on 
physical  data  via  a  process  of  elimination,  much  like  the  candidate  elimination  algorithm  of  Mitchell 
[56,  57].  The  theory  concerns  the  feedback  control  configuration  in  Figure  1.  As  always  in  control 
theory,  the  goal  is  to  determine  a  control  law  K  for  the  plant  P  such  that  the  closed-loop  system 
response,  say  T,  satisfies  given  specifications.  Unfalsified  control  theory  is  concerned  with  the  case  in 
which  the  plant  is  either  unknown  or  is  only  partially  known  and  one  wishes  to  fully  utilize  information 
from  measurements  in  selecting  the  control  law  K.  In  the  theory  of  unfalsified  control,  learning  takes 
place  when  new  information  in  measurement  data  enables  one  to  eliminate  from  consideration  one  or 
more  candidate  controllers. 

As  indicated  in  Fig.  2  three  elements  that  define  the  unfalsified  control  problem  are  (1)  plant 
measurement  data,  (2)  a  class  of  candidate  controllers,  and  (3)  a  performance  specification,  say  Tspec, 
consisting  of  a  set  of  admissible  3-tuples  of  signals  (r,y,u).  More  precisely,  we  have  the  following. 


Definition  [31]  A  controller  K  is  said  to  be  falsified  by  measurement  information  if  this  information 
is  sufficient  to  deduce  that  the  performance  specification  (r,  y ,  u)  G  Tspec  Vr  G  1Z  would  be  violated  if 
that  controller  were  in  the  feedback  loop.  Otherwise,  the  control  law  K  is  said  to  be  unfalsified.  □ 

To  put  plant  models,  data  and  controller  models  on  an  equal  footing  with  performance  specifica¬ 
tions,  these  like  Tspec  are  regarded  as  sets  of  3-tuples  of  signals  (r,  y,u)  —  that  is,  they  are  regarded 
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Supervisor 

Adaptive  Unfalsified 

Switching  Control 

Figure  1:  An  unfalsified  adaptive  controller  has  a  two-tier  structure,  consisting  of  a  adaptive  supervisor  and  a  conven¬ 
tional  controller  K.  The  supervisor  monitors  plant  data  (u,y)  for  evidence  that  would  falsify  candidate  controllers.  If 
the  currently  active  controller  K  becomes  falsified  by  data,  then  an  as  yet  unfalsified  controller  is  switched  into  the 
loop  to  replace  it. 


as  relations  in  1Z  x  y  x  U.  For  example,  if  P  :  U  —>  y  and  K  :  1Z  x  y  —>  U  then 


P 

K 


{  (r,  y,u)  \y  =  Pu  } 


(■ r,y,u ) 


u  =  K 


r 

y 


And,  if  J(r,y,u,t)  is  a  given  loss- function  that  we  wish  to  be  non-positive  for  all  time  £,  then  the 
performance  specification  Tspec  would  be  simply  the  set 

Tgpec  =  {  (r,  y,  u)  |  J (r,  y,  u,  t)  <  0  Vi  }  .  (1) 

On  the  other  hand,  experimental  information  from  a  plant  corresponds  to  partial  knowledge  of  the 
plant  P.  Loosely,  data  may  be  regarded  as  providing  a  sort  of  an  “interpolation  constraint”  on  the 
graph  of  P —  i.e. ,  a  ‘point’  or  set  of  ‘points’  through  which  the  infinite-dimensional  graph  of  dynamical 
operator  P  must  pass. 

Typically,  the  available  measurement  information  will  depend  on  the  current  time,  say  r.  For 
example,  if  we  have  complete  data  on  (u,  y )  from  time  0  up  to  time  r  >  0,  then  the  measurement 
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information  is  characterized  by  the  set  [31] 

( u  —  Udata) 
( V  Udata) 


Pdata  =  \  {r,y,u)  eUxU  xy 


(2) 


where  PT  is  the  familiar  time-truncation  operator  of  input-output  stability  theory  (cf.  [58,  59]),  viz., 


[  0,  otherwise. 


The  main  result  of  unfalsified  control  theory  is  the  following  theorem  which  gives  necessary  and 
sufficient  conditions  for  past  open-loop  plant  data  Pdata  to  falsify  the  hypothesis  that  controller  K  can 
satisfy  the  performance  specification  Tspec. 


Unfalsified  Control  Theorem  [31]  A  control  law  K  is  unfalsified  by  measurement  information 
Pdata  if,  and  only  if,  for  each  triple  (ro,?/cb^o)  G  Pdata  H  K,  there  exists  at  least  one  pair  (uo,yo) 
such  that 

(TO  ,  VO  5  ^  Pdata  H  K  fl  Tspec-  (3) 

Proof:  With  controller  K  in  the  loop,  a  command  signal  ro  G  1Z  could  have  produced  the  measurement 
information  if,  and  only  if,  (ro,yo,uo)  G  Pdata  H  K  for  some  (uo,yo)-  The  controller  K  is  unfalsified 
if  and  only  if  for  each  such  ro  there  is  at  least  one  ( possibly  different)  pair  {u\,yi)  which  also  could 
have  produced  the  measurement  information  with  K  in  the  loop  and  which  additionally  satisfies  the 
performance  specification  ( ro,yi,u\ )  G  Tspec.  That  is,  K  is  unfalsified  if  and  only  if  for  each  such  ro, 
condition  (3)  holds.  □ 


The  Unfalsified  Control  Theorem  constitutes  a  mathematically  precise  statement  of  what  it 
means  for  experimental  data  and  a  performance  specification  to  be  inconsistent  with  a  particular 
controller.  It  has  some  interesting  implications: 

•  The  Unfalsified  Control  Theorem  is  nonconservative;  i.e.,  it  gives  “if  and  only  if”  conditions 
on  K.  It  uses  all  the  information  in  the  past  data  —  and  no  more.  It  provides  a  mathemat¬ 
ically  precise  “sieve”  which  rejects  any  controller  which,  based  on  experimental  evidence,  is 
demonstrably  incapable  of  meeting  a  given  performance  specification. 

•  The  Unfalsified  Control  Theorem  is  “model  free”.  No  plant  model  is  needed  to  test  its 
conditions.  There  are  no  assumptions  about  the  plant. 

•  Information  Pdata  which  invalidates  a  particular  controller  K  need  not  have  been  generated  with 
that  controller  in  the  feedback  loop;  it  may  be  open  loop  data  or  data  generated  by  some  other 
control  law  (which  need  not  even  be  in  K). 

•  When  the  sets  Pdata ,  K  and  Tspec  are  each  expressible  in  terms  of  equations  and/or  inequalities, 
then  falsification  of  a  controller  reduces  to  a  minimax  optimization  problem.  For  some  forms  of 
inequalities  and  equalities  (e.g.,  linear  or  quadratic),  this  optimization  problem  may  be  solved 
analytically,  leading  to  procedures  for  direct  identification  of  controllers  —  as  the  example  in 

[34]. 
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Figure  2:  An  unfalsification  process  is  used  supervisory  control  design.  The  process  requires  three  types  of  input 
(1)  goals,  (2)  candidate  controllers  and  (3)  data.  Controllers  are  sifted  to  find  those  that  are  consistent  with  both 
performance  goals  and  physical  data.  No  plant  models  are  required  while  the  process  is  running,  though  a  plant  model 
can  be  useful  for  prior  selections  of  the  candidate  controllers  and  the  performance  goal. 


•  Given  data  (t/o?yo)  and  a  candidate  controller  A,  the  ro’s  satisfying  the  conditions  of  the  Un¬ 
falsified  Control  Theorem  are  called  the  fictitious  reference  signals.  When  K  has  a  causal 
inverse,  the  tq  is  uniquely  determined  by  (i/o>2/o)  and  a  candidate  controller  K\  that  is,  there 
exists  a  causal  function  r(K,uo,yo)  such  that 

r0  =  r(K,  u0,yo). 

The  function  f(lf,  izo,J/o)  is  called  the  fictitious  reference  signal  [27,  31];  and  is  closely  related  to 
the  virtual  reference  signal  of  [50] . 

•  In  adaptive  control  the  fact  that  the  supervisor  chooses  a  controller  means,  at  least  implicitly, 
that  there  is  a  real- valued  data-driven  cost  function  V(K,yo,uo,t)  such  that  at  any  give  time  r 
the  active  controller  is  cost-minimizing 

K(y0,u0,t)  =  arg  min  V(K,y0,u0,t).  (4) 

Ke  K 

In  the  case  of  unfalsified  control,  the  cost  function  V  is  determined  from  the  cost  J(r,  y,  u,  t )  and 
evaluating  it  with  r  equal  to  the  fictitious  reference  signal  r(K,  uo,yo): 

V(K,y0,u0,t)  =  J(r(K,u0,yo),yo,uo,t).  (5) 

2.2  Data-Driven  Learning  and  Adaptive  Control 

The  unfalsified  control  theorem  says  simply  that  controller  falsification  can  be  tested  by  computing 
an  intersection  of  certain  sets  of  signals.  A  noteworthy  feature  of  the  unfalsified  control  theory  is  that 
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Table  1:  Recursive  Adaptive  Control  Algorithm 

Input: 

•  A  finite  set  K  of  m  candidate  dynamical  controllers  Ki(r,y,u)  =  0,  (i  =  1, . . . ,  m)  each  having  the  causal-left- 
invertibility  property  that  r(t )  is  uniquely  determined  from  Ki(r,y,u)  =  0  by  past  values  of  u(t),y(t). 

•  Sampling  interval  At  and  current  time  r  =  nAt ; 

•  Plant  data  (u(t),y(t)),  t  G  [0, r]; 

•  Performance  specification  set  Tspec  consisting  of  the  set  of  triples  (r,y,u)  satisfying  for  all  k  =»  1, . . .  ,  n 

rkAt 

/  Tspec(r(t),y(t),u(t),t)  dt  <  0. 

Jo 

Initialize: 

set  k  =  0,  set  i  —  m\ 

for  i  =  0  :  m,  set  s(i)  =  1,  set  J(i)  =  0,  end. 

Procedure: 

while  i  >  0; 

k  =  k  +  1; 
for  i  =  1  :  ra ; 

if  s(i)  >  0; 

for  each  t  G  [(k  —  1)A t,  kAt]\ 
solve  Ki(r,y,u)  —  0  for  r(t); 

(note  that  r(t)  exists  and  is  unique  since  Ki  has  the  causal-left- 
invertibility  property) 
end; 

J(i)  =  J(i )  +  /(^i)At  APec(r(t),2/(t),u(f),f)  dt\ 
if  J(i)  >  0,  set  s(z)  =  0,  end; 

end; 

end; 

i  =  max  {  i  |  s(i)  >  0  }; 

end. 


a  controller  need  not  be  in  the  loop  to  be  falsified.  Broad  classes  of  controllers  can  be  falsified  with 
open-loop  plant  data  or  even  data  acquired  while  other  controllers  were  in  the  loop.  Adaptive  control 
is  achieved  within  the  this  framework  by  using  the  unfalsification  process  as  the  key  element  of  a 
supervisory  controller  (cf.  [60,  61]).  The  supervisor  switches  an  unfalsified  controller  into  the  feedback 
loop  whenever  the  current  controller  in  the  loop  is  amongst  those  falsified  by  observed  plant  data  - 
see  Fig.  2.  The  supervisor  chooses  as  the  current  control  law  one  that  is  not  falsified  by  the  past  data, 
resulting  in  a  control  law  that  is  adaptive  in  the  sense  that  it  learns  in  real  time  and  changes  based 
on  what  it  learns. 

Like  the  controllers  of  [62,  63],  this  approach  to  adaptive  real-time  unfalsified  control  leads  to  a  sort 
of  “switching  control.”  Controllers  which  are  determined  to  be  incapable  of  satisfactory  performance 
are  switched  out  of  the  feedback  loop  and  replaced  by  others  which,  based  on  the  information  in 
past  data,  have  not  yet  been  found  to  be  inconsistent  with  the  performance  specification.  However, 
adaptive  unfalsified  controllers  generally  would  not  be  expected  to  exhibit  the  poor  transient  response 
associated  with  switching  methods  such  as  [62].  The  reason  is  that,  unlike  the  theory  in  [62],  unfalsified 
control  theory  efficiently  eliminates  broad  classes  of  controllers  before  they  are  ever  inserted  in  the 
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Figure  3:  A  data-driven  unfalsified  missile  controller  would  have  abilities  to  adaptively  discover  solutions  in  real-time 
to  compensate  for  sudden  in-flight  changes  and  damage. 


feedbackunfalsified  control  and  other  adaptive  methods  is  that  in  unfalsified  control  one  evaluates 
candidate  controllers  objectively  based  on  experimental  data  alone,  without  prejudicial  assumptions 
about  the  plant. 

While,  in  principle,  the  unfalsified  control  theory  allows  for  the  set  K  to  include  continuously 
parameterized  sets  of  controllers,  restricting  attention  to  candidate  controller  sets  K  with  only  a  finite 
number  of  elements  can  simplify  computations.  Further  simplifications  result  by  restricting  attention 
to  candidate  controllers  that  are  “causally-left-invertible”  in  the  sense  that,  given  a  K  E  K,  the  current 
value  of  r(t)  is  uniquely  determined  by  past  values  of  u(t),y(t).  When  (2)  holds,  these  restrictions  on 
Tspec  and  K  are  sufficient  to  permit  the  unfalsified  set  to  be  evaluated  in  real-time  via  the  following 
conceptual  in  Table  1. 

This  algorithm  returns  for  each  time  the  least  index  i  for  which  Kj  is  unfalsified  by  the  past 
plant  data.  Real-time  unfalsified  adaptive  control  is  achieved  by  always  taking  as  the  currently  active 
controller 


provided  that  the  data  does  not  falsify  all  candidate  controllers.  In  this  latter  case,  the  algorithm 
terminates  and  returns  i  —  0. 

It  is  important  to  note  that  while  the  above  algorithm  is  geared  towards  the  case  of  an  integral 
inequality  performance  criterion  Tspec  and  a  finite  set  of  causally-left-invertible  Kf  s,  the  underlying 
theory  is,  in  principle,  applicable  to  arbitrary  non-finite  controller  sets  K  and  to  hybrid  systems  with 
both  discrete  and  continuous  time  elements. 


Comment  If  the  plant  is  slowly  time-varying ,  then  older  data  ought  to  be  discarded  before  evaluating 
controller  falsification.  This  may  be  effected  within  the  context  of  our  the  Recursive  Adaptive  Control 
Algorithm  described  in  Table  1  by  fixing  r  =  tq  and  regarding  t  —  To  as  the  deviation  from  the  current 
time.  The  result  is  a  an  algorithm  which  only  considers  data  from  moving  time-window  of  fixed  duration 
To  time-units  prior  to  the  current  real-time.  In  this  case  the  unfalsified  controller  set  K ok  no  longer 
shrinks  monotonically  as  it  would  if  r  were  increasing  in  lockstep  with  real-time. 
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2.3  Design  Studies 

Design  studies  have  confirmed  the  theoretical  expectation  that  supervisory  controllers  that  are  designed 
based  on  the  logic  of  unfalsification  can  be  effective  in  closing  the  outer  data-driven  loop  on  the  control 
design  process.  Unfalsified  control  theory  has  proved  effective  in  applications  involving  both  off-line 
controller  gain  tuning  and  in  real-time  adaptive  control  design  studies.  Following  is  a  brief  description 
of  some  of  the  design  studies  that  have  helped  us  to  better  understand  the  potential  of  the  unfalsified 
control  theory,  as  well  as  limitations  of  the  current  theory. 


2.3.1  Missile  Autopilot 

One  design  study  that  we  conducted  involved  using  an  unfalsified  controller  to  robustly  discover  PID 
controller  gains  for  an  adaptive  missile  autopilot  ‘on  the  fly’  in  real-time  [36].  Figure  3  summarizes  the 
results  of  the  missile  design.  In  all  trials,  the  response  of  the  adaptive  loops  was  swift  and  sure-footed 
—  in  stark  contrast  to  what  would  be  expected  from  traditional  quasi-static  adaptive  methods  (e.g., 
standard  model  reference  adaptive  control). 


2.3.2  Universal  PID  Controller 


One  application  of  the  theory  involved  implementing  a  PID-based  adaptive  ‘universal’  controller  im¬ 
plemented  as  Matlab  Simulink  block  based  on  the  unfalsified  theory  [37]  —  see  Fig  4.  The  controller 
sifts  through  a  bank  of  candidate  controllers  in  real-time,  identifying  which  of  the  30  controllers  is 
unfalsified  with  respect  to  an  inequality  performance  goal  of  ‘mixed-sensitivity’  type,  viz., 

J(r,  y,u,t)  <  1  for  all  t 


where 


J(r,  ?/,  u,  t )  =  max 

T<t 


\\wi  *  (r  —  y)\\2  +  ||^2  *  u\\2  —  <j2t 


+  P 


(6) 


and  w\  and  w>2  are  ‘weighting’  filters  and  p  and  sigma  are  constants  chosen  by  the  designer  based 
on  control  bandwidth  and  robustness  considerations  -  exactly  as  in  standard  mixed-sensitivity  robust 
control  design.  Initially,  there  are  30  candidate  PID  gain  combinations,  indicated  on  the  vertical  axis. 
Unfalsified  controllers  are  indicated  by  the  horizontal  traces.  When  the  currently  active  controller 
indicated  by  the  bold  trace  becomes  falsified,  then  one  of  the  as  yet  unfalsified  controllers  is  switched 
into  the  loop  to  replace  it.  Notice  that  adaptive  supervisor  loop  is  so  fast  that  the  controller  is 
able  to  stabilize  the  open-loop  unstable  plant  without  prior  plant  knowledge  and  without  appreciable 
transients.  The  unfalsification  procedure  always  discovers  a  stabilizing  controller,  The  supervisory 
controller  designed  via  unfalsified  control  quickly  discovers  a  stabilizing  candidate  controller  for  the 
open-loop-unstable  plant.  The  unfalsified  control  theory  assures  the  procedure  will  always  converge 
to  a  stabilizing  controller  that  meets  the  performance  goals,  provided  only  that  the  initial  candidate 
controller  set  contains  at  least  one  such  controller;  no  other  prior  knowledge  of  the  plant  is  necessary  to 
assure  convergence.  In  particular,  the  unfalsification  logic  in  the  supervisor  is  guaranteed  to  stabilize 
and  meet  performance  goals  without  any  of  the  ’standard  assumptions’  on  the  plant  (e.g.,  [64,  65])  — 
viz.,  without  minimum  phase,  linearity,  bounds  on  relative-degree,  knowledge  of  the  sign  of  the  plant 
high  frequency  gain,  or  persistence  of  excitation. 


2.3.3  Robot  Manipulator  Arm 

We  used  to  unfalsified  methodology  to  adaptively  tune  the  parameters  of  a  nonlinear  ‘computed-torque’ 
controller  for  a  robot  manipulator  arm  [66,  32]  —  see  Fig  5.  The  arm  proved  to  be  capable  of  a  quick 
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Figure  4:  Here  we  see  a  Matlab  Simulink  time-history  for  an  adaptive  ‘universal  PID  controller’. 


and  reliable  control  response  despite  large  and  sudden  variations  in  load  mass.  Again,  the  controller 
performed  with  precision,  despite  noise,  dynamical  actuator  uncertainties  and  without  prior  knowledge 
of  the  plant  model  or  its  parameters.  Results  for  the  robot  design  were  surefooted  and  precise,  with 
the  controller  maintaining  an  order  of  magnitude  more  precise  control  than  a  similar  model-reference 
adaptive  controller  during  widely  fluctuating  manipulator  load  variations;  the  controller  was  also  more 
robust  in  that  it  was  capable  of  maintaining  precise  control  even  during  load  variations  that  destabilized 
a  similarly  structured  model-reference  adaptive  controller. 

2.3.4  Industrial  Process  Control 

Although  very  few  researchers  other  than  ourselves  have  as  yet  examined  unfalsified  control  meth¬ 
ods,  those  who  have  taken  this  step  have  predictably  confirmed  the  effectiveness  of  unfalsified  control 
methods  in  several  industrial  process  control  applications.  For  example,  Kosut  [38]  examined  unfalsi¬ 
fied  controller  for  direct  data-driven  off-line  control  gain  tuning  under  the  assumption  of  a  noise-free 
linear-time-invariant  plant.  Woodley,  How  and  Kosut  [67]  and  used  the  theory  with  good  result  for 
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Figure  5:  Unfalsified  control  produced  superior  results  for  a  nonlinear  two-link  robot  manipulator  subject  to  uncertain 
dynamics,  noisy  disturbances  and  abrupt  changes  in  load  mass.  The  two  sluggishly  smooth  traces  large  amplitude 
signals  in  the  plot  are  with  a  conventional  adaptive  controller  used  to  adjust  control  gain-vector  0(t),  and  the  two  very 
low  amplitude  traces  are  for  the  unfalsified  controller.  The  unfalsified  controller  had  a  much  quicker,  sure-footed  and 
precise  response  without  increased  control  effort. 


data-driven  discovery  of  good  control  gains  for  a  laboratory  control  problem  involving  two  spring- 
connected  masses.  Also,  Collins  and  Fan  [39]  successfully  used  the  unfalsified  control  methodology  in 
a  run-to-run  setting  to  tune  gains  off-line  in  an  industrial  weigh-belt  feeder  control  design  study.  More 
recently,  there  have  been  some  promising  adaptive  control  applications  to  machine  control  by  Razavi 
and  Kurfess  [40,  68]  based  on  the  unfalsified  control  methodology. 


3.0  STABILITY  AND  CONVERGENCE 

When  modeling  assumptions  about  the  plant  fail  to  hold,  there  is  the  possibility  that  badly  designed 
adaptive  algorithms  can  fail  to  stabilize  —  even  when  the  adaptive  control  problem  is  theoretically 
feasible  in  the  sense  that  one  of  the  candidate  controllers  K  G  K  is  stabilizing.  The  problem  is  that  in 
most  cases,  ‘proofs’  of  stability  in  adaptive  control  only  hold  when  there  is  no  mismatch  between  model 
assumptions  and  the  true  plant.  Well-know  standard  assumptions  in  adaptive  control  include  upper- 
bounds  on  plant  order,  assumption  the  plant  is  minimum  phase  and  has  no  time-delays,  or  that  the 
plant  is  ‘sufficiently  close’  to  one  of  several  presumed  prior  plant  models.  There  are  a  number  of  studies 
illustrating  how  adaptive  systems  can  fail  in  the  presence  of  a  mismatch  between  assumptions  and 
reality  [69,  70].  A  typical  result  of  model  mismatch  instability  is  shown  in  Figure  6which  illustrates  the 
consequences  of  model  mismatch  instability  for  a  typical  multi-model  adaptive  (MM AC)  system.  But, 
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It  crashes. 
Why? 


Figure  6:  Without  cost-detectability,  model  mismatch  can  cause  an  adaptive  controller  to  switch  a  destabilizing 
controller  C2  into  the  loop  and  keep  it,  even  when  the  original  stabilizing  controller  C\  was  working  well  [70]. 


though  currently  popular  adaptive  algorithms  are  susceptible  to  model-mismatch  instability,  optimal 
unfalsified  adaptive  design  designs  with  a  suitable  cost-detectability  properties  as  well  as  some  lesser 
known  early  adaptive  methods  robustly  avoid  model  mismatch  instability,  provided  that  the  unfalsified 
cost  function  J(r,y,u,t)  is  chosen  to  have  a  property  called  cost  detectability  [72,  73]. 

For  example,  the  multi-controller  adaptive  control  (MCAC)  switching  algorithms  and  Martens- 
son  [71]  and  of  Fu  and  Barmish  [62]  are  robust  against  model-mismatch.  They  require  essentially 
only  feasibility  to  assure  convergence.  The  work  robustly  in  the  presence  of  model  mismatch  because 
the  assign  a  cost  penalty  to  destabilizing  controllers  that  tends  to  infinity  for  destabilizing  candidate 
controllers.  This  is  the  essential  feature  of  cost-detectability.  Cost-detectability  is  a  feature  not  typi¬ 
cally  present  for  most  currently  popular  adaptive  algorithms,  and  it  explains  why  they  are  susceptible 
to  model  mismatch  instability.  Without  the  cost  detectability  property,  adaptive  control  algorithms 
are  in  general  not  able  to  reliably  distinguish  stable  and  unstable  behavior  when  model-mismatch  ex¬ 
ceeds  certain  thresholds.  While,  unfortunately,  the  early  cost-detectable  algorithms  of  [71,  62]  are  too 
slow  for  applications  requiring  real-time  adaptive  stabilization,  this  is  not  true  of  unfalsified  adaptive 
control. 

Definition  3.1  (Cost  Detectablility)  A  cost  function  V(K,u,y,t)  is  said  to  have  the  cost  de¬ 
tectability  property  if  it  has  all  the  following  attributes: 

1.  V (K,  u ,  y,  t)  is  a  monotone  increasing  function  of  time  t  for  all  K ,  u ,  y, 

2.  When  K  is  not  stabilizing  and  the  data  u,y  are  not  stable,  then  00  =  lim^oo  V(K,u,y,t). 

3.  When  K  is  stabilizing,  lim^oo  V(K,  u,  y,  t)  is  uniformly  bounded  for  all  (u,y,t),  ( even  if  the 
data  u,y  are  unstable  signals). 

So-called  cL2e-gain  related’  cost  functions  like  the  ‘mixed-sensitivity’  cost  (6)  generically  assure  the 
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cost  detectability  property  for  unfalsified  cost  function  [72,  73] 

V(K,y,u,t)  =  J(r(K,u,y),y,u,t). 

The  plant  does  not  need  to  be  minimum  phase,  nor  does  it  need  to  satisfy  any  other  standard  as¬ 
sumptions.  So,  with  L2e-gain  related  cost  functions,  destabilizing  controllers  are  never  retained  when  a 
stabilizing  candidate  controller  is  available,  irrespective  of  model  mismatch  and  irrespective  of  whether 
standard  assumptions  or  other  prior  beliefs  about  the  plant  fail  to  hold.  Model-mismatch  instabil¬ 
ity  cannot  occur.  The  adaptive  system  is  stable  whenever  the  adaptive  control  problem  is  feasible. 
[72,  73].  Moreover  unlike  the  early  cost-detectable  adaptive  algorithms  of  Martensson  [71]  and  of 
Fu  and  Barmish  [62],  unfalsified  adaptive  control  systems  adapt  with  optimally  rapidity.  Simulation 
studies  [37]  demonstrate  that  unfalsified  adaptive  systems  can  adapt  fast  enough  to  do  real-time  sta¬ 
bilization  of  open-loop  unstable  plants  in  cases  where  the  measurement  signal  to  noise  ratio  is  not  too 
great. 

Using  the  hysteresis  switching  lemma  [74],  Stefanovic  et  al.  proved  the  following. 

Convergence  and  Stability  [72]  A  switched  sequence  of  controllers  K(tf)  (i  =  1,2,. . .)  that  mini¬ 
mize  the  current  unfalsified  cost  V ( K ,  u ,  y ,  t)  at  each  switch-time  t{  will  stabilize  the  plant  P  if  the  cost 
V(K,u,y,t )  cost  detectability  property.  Proof:  As  illustrated  in  Figure  7,  cost  detectability  assures 
that  the  cost-minimizing  controller  tends  towards  one  with  finite  cost,  which  implies  stability  when  the 
cost  has  the  cost- detectability  property.  See  [72]  for  more  a  detailed  proof.  □ 

While  Morse  et  al.  were  able  to  demonstrate  the  cost-detectability  property  (which  they  called 
‘tunability’)  for  the  adaptive  methods  that  they  examined  only  by  introducing  assumptions  on  the 
plant,  the  results  of  [72,  73]  demonstrated  one  can  directly  design  an  unfalsified  cost  V(K,  uo,  yo,t)  to 
be  cost  detectable  without  regard  to  plant  assumptions.  Cost-detectably  unfalsified  adaptive  control 
selectors  quickly,  reliably  and  robustly  sift  candidate  controllers  without  assumptions  on  the  plant 
itself. 

In  designs  like  [37],  we  have  demonstrated  that  the  unfalsified  control  approach  with  an  L2e-gain 
related  cost  function  converges  quickly  and  reliably  in  real  time  to  a  stabilizing  controller  that  robustly 
achieves  specified  performance  goals,  often  converging  within  a  fraction  of  an  unstable  plant’s  fastest 
unstable  time  constant.  This  speed  of  adaptive  response  means  that  “bursting  phenomena”  that 
plague  conventional  slow  adaptive  systems  do  not  occur,  even  in  the  absence  of  persistently  exciting 
disturbance  signals.  Because  our  unfalsified  adaptive  systems  perform  reliably  irrespective  of  plant 
model  mismatch,  they  have  the  potential  to  reliably  achieve  rapid  real-time  failure  recovery  for  battle- 
damaged  aircraft  and  similar  systems. 

4.0  DISCUSSION 

In  the  unfalsified  control  approach  to  supervisory  control,  decisions  to  adapt  are  data-driven.  Deter¬ 
mination  of  which  candidate  control  laws  are  suitable  are  made  based  on  experimental  evidence,  i.e. , 
the  actual  values  of  sensor  output  signals  and  actuator  input  signals.  In  this  process  the  role,  if  any,  of 
plant  models  and  of  probabilistic  hypotheses  about  stochastic  noise  and  random  initial  conditions  is 
entirely  an  a  priori  role:  These  provide  concepts  which  are  useful  in  selecting  the  class  K  of  candidate 
controllers  and  in  selecting  achievable  goals  (i.e.,  selecting  Tspec ).  The  methods  of  traditional  model- 
based  control  theories  (root  locus,  stochastic  optimal  control,  Bode-Nyquist  theory,  robust  control 
and  so  forth)  provide  mechanizations  of  this  prior  selection  and  narrowing  process.  Unfalsified  control 
takes  over  where  traditional  model-based  methods  leave  off,  providing  a  mathematical  framework  for 
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Figure  7:  l_2e-gain  related  cost  functions  have  the  cost-detectability  property:  They  are  monotone  in  t,  uniformly 
bounded  for  stabilizing  K  and  unbounded  for  destabilizing  K.  Cost-minimizing  adaptive  laws  robustly  converge  irre¬ 
spective  of  the  size  of  plant  model-mismatch  [72,  73]. 


determining  the  proper  consequences  of  experimental  observations  on  the  choice  of  control  law.  In 
effect,  the  theory  gives  one  a  model- free  mathematical  “sieve”  for  candidate  controllers,  enabling  us 
(i)  to  precisely  identify  what  of  relevance  to  attaining  the  specification  Tspec  can  be  discovered  from 
experimental  data  alone  and  (ii)  to  clearly  distinguish  the  implications  of  experimental  data  from 
those  of  assumptions  and  other  prior  information. 

The  Unfalsified  Control  Theorem  explains  the  learning  mechanisms  of  adaptive  control  theory. 
It  provides  an  exact  characterization  of  what  can,  and  what  cannot,  be  learned  from  experimental 
data  about  the  ability  of  a  given  class  of  controllers  to  meet  a  given  performance  specification.  A 
salient  feature  of  the  theory  is  that  the  data  used  to  falsify  a  class  of  control  laws  may  be  either 
open-loop  data  or  data  obtained  with  other  controllers  in  the  feedback  loop.  Consequently,  large 
classes  of  candidate  controllers  are  falsified  by  even  a  few  experimental  samples  of  plant  input  output 
data.  Candidate  controllers  need  not  be  actually  inserted  in  the  feedback  loop  to  be  falsified.  This  is 
important  because  it  means  that  adaptive  unfalsified  controllers  will  be  significantly  less  susceptible 
to  poor  transient  response  than  adaptive  learning  algorithms  which  require  inserting  controllers  in  the 
loop  one-at-a-time  to  determine  if  they  are  unsuitable. 

A  noteworthy  feature  of  the  unfalsified  control  theory  is  its  flexibility  and  simplicity  of  implemen¬ 
tation.  Controller  falsification  typically  involves  only  real-time  integration  of  algebraic  functions  of  the 
observed  data,  with  one  set  of  integrators  for  each  candidate  controller.  The  theory  may  be  readily 
applied  to  nonlinear  time- varying  plants,  as  well  as  to  linear  time-invariant  ones. 


5.0  CONCLUSION 

As  robust  control  theory  has  matured,  a  key  challenge  has  been  the  need  for  a  more  flexible  theory 
that  provides  a  unified  basis  for  representing  and  exploiting  evolving  real-time  data.  The  role  of 
unfalsified  control  is  to  close  the  loop  on  the  adaptive  and  robust  control  design  processes  by  developing 
precise  data-driven  methods  to  implement  supervisory  control  schemes  that  can  optimally  exploit  the 
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information  in  data  to  enhance  robustness  and  improve  performance.  The  types  of  control  designs 
will  be  better  able  to  compensate  for  uncertain  and  time-varying  effects,  battle  damage,  equipment 
failures  and  other  changing  circumstances. 

A  salient  feature  of  unfalsified  adaptive  control  systems  is  that  they  are  robustly  convergent  ir¬ 
respective  of  model  mismatch,  provided  that  the  problem  is  feasible  and  one  uses  a  cost  function 
J(r,y,u,t)  with  the  cost-detectability  property.  No  plant  assumptions  are  required  to  assure  conver¬ 
gence.  Safe  adaptive  control  is  assured  whenever  the  adaptive  control  problem  is  feasible.  Adaptation 
is  rapid  and  reliable  because  unfalsified  control  makes  optimal  use  of  information  in  measurements 
data  to  eliminate  destabilizing  controllers  efficiently  and  quickly,  selecting  only  controllers  that  are 
optimal  with  respect  to  data. 

“It  is  a  capital  mistake  to  theorize  before  one  has  data .  Insensibly  one  begins  to  twist  facts 
to  suit  theories  instead  of  theories  to  suit  facts.  ” 

Sherlock  Holmes 
Arthur  Conan  Doyle 
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